As a signal interpolation technique using a correlation between component signals that constitute image signals, a technique of improving accuracy in a pixel interpolation method in a single primary color image sensor has been proposed (e.g., see Non-Patent Literature 1). The interpolation technique of the image signal in such an image sensor is developed to interpolate RGB signals (R: red signal, G: green signal, B: blue signal) in an RGB color space. Thus, signal degradation caused by encoding is not considered.
As a signal interpolation technique focusing on a difference in sampling frequency of a YUV signal in a YUV color space, a chrominance signal interpolation technique of a format color image has been proposed (e.g., see Non-Patent Literature 2). In this technique, highly accurate interpolation is performed by generating an interpolation signal of a chrominance signal (i.e., a U signal=B−Y, a V signal=R−Y) using the height of a sampling frequency of a luminance (Y) signal. Such a signal interpolation technique focusing on a difference in a sampling frequency of a YUV signal is also developed to interpolate YUV signals. Thus, signal degradation caused by encoding is not considered.
These signal interpolation techniques are suitable to interpolate pre-decoded image signals in encoding image signals by an irreversible encoding method (e.g., MPEG-2 and H.264) but not suitable to interpolate image signals after encoding. For example, when a YUV signal is encoded by an irreversible encoding process, degradation of a luminance signal propagates to a chrominance signal based on this luminance signal as the luminance signal degrades. Since these signal interpolation techniques are not processes to reduce degradation itself of the luminance signal, degradation of the luminance signal is not reduced.
Various deblocking filters (e.g., a deblocking filter in, for example, H.264) are proposed to reduce degradation in encoding. These deblocking filters process each of image signal components independently so that degradation does not become visible. Thus, it is not necessary to reduce degradation of an original image signal after encoding.
In a related art irreversible encoding method (e.g., H.264), as a luminance signal intra-frame prediction (intra-frame prediction) method, an original signal is predicted by extrapolation using pixel information obtained by decoding a closely located encoded block which approaches as signal prediction of an encoded block, a prediction signal is generated, and a difference signal between the original signal and the prediction signal is encoded. This prediction processing is separately performed to a signal stream of each component of the three component signals on the presupposition that three component signals has no correlation among then because of their low correlation. When the signals are seen locally, however, fluctuations of signals between each component signal has a correlation, and signals can be predicted mutually. This correlation is not used in the related art encoding methods.
For example, as shown in FIG. 9, a related art encoding device 100 (e.g., an encoding device for H.264) has an encoded signal stream of a first component signal (a U signal or a V signal) and an encoded signal stream of a second component signal (a Y signal). These encoded signal streams are provided with rearrangers 12-1 and 12-2, subtracters 13-1 and 13-2, orthogonal transformers 14-1 and 14-2, quantizers 15-1 and 15-2, variable length encoders 16-1 and 16-2, inverse quantizers 17-1 and 17-2, inverse orthogonal transformers 18-1 and 18-2, memories 21-1 and 21-2, inter-frame/intra-frame predictors 22-1 and 22-2, adders 23-1 and 23-2, and a bit stream forming unit 25 that reconstructs the encoded signal of each encoded signal stream into a bit stream to be sent outside.
FIG. 9 illustrates an example in which orthogonal transformation and quantization processing are performed in parallel for each component signal, but it is also possible to perform an encoding process while sequentially reading each component signal out. In a case in which the component signal consists of a YUV signal, in performing orthogonal transformation and quantization processing, the U signal or the V signal is subject to orthogonal transformation and quantization via the orthogonal transformer 14-1 and the quantizer 15-1, respectively and, similarly, the Y signal is subject to orthogonal transformation and quantization via the orthogonal transformer 14-2 and the quantizer 15-2, respectively. Regarding a local decoding process, a signal stream switcher (not illustrated) for switching between the orthogonal transformers 14-1 and 14-2 and between the quantizers 15-1 and 15-2 may be provided so that each component signal can be sequentially read out and processed. Hereinafter, as a typical example, an inter-frame prediction operation and an intra-frame prediction operation of the encoded signal stream of the first component signal (the U signal or the V signal) are described sequentially.
[Inter-Frame Prediction]
The rearranger 12-1 makes rearrangement for the encoding the first component signal per pixel block of small areas, and sends the signal to the subtracter 13-1 and the inter-frame/intra-frame predictor 22-1.
With respect to an original signal of the first component signal supplied from the rearranger 12-1, the inter-frame/intra-frame predictor 22-1 performs motion vector detection using a reference image obtained from the memory 21-1, performs motion compensation using the obtained motion vector, and then outputs the obtained prediction signal to the adder 23-1 and the subtracter 13-1. Information about the motion vector is sent to the variable length encoder 16-1.
The subtracter 13-1 generates a difference signal between the original signal from the rearranger 12-1 and the prediction signal from the inter-frame/intra-frame predictor 22-1, and sends the generated difference signal to the orthogonal transformer 14-1.
With respect to the difference signal supplied from the subtracter 13-1, the orthogonal transformer 14-1 performs orthogonal transformation (e.g., DCT) for every pixel block of a small area and sends the signal to the quantizer 15-1.
The quantizer 15-1 selects a quantization table corresponding to a pixel block of a small area supplied from the orthogonal transformer 14-1 and performs quantization processing, sends the quantized signal to the variable length encoder 16-1 and, at the same time, sends the quantized signal to the inverse quantizer 17-1.
The variable length encoder 16-1 scans the quantized signal supplied from the quantizer 15-1, performs a variable length encoding process to generate a bit stream. At the same time, the variable length encoder 16-1 performs variable length encoding on the information about the motion vector supplied from the inter-frame/intra-frame predictor 22-1 and outputs the information.
The inverse quantizer 17-1 performs an inverse quantization process about the quantized signal supplied from the quantizer 15-1, and outputs the signal to the inverse orthogonal transformer 18-1.
The inverse orthogonal transformer 18-1 performs inverse orthogonal transformation (e.g., IDCT) to an orthogonal transformation coefficient supplied from the inverse quantizer 17-1, and outputs the coefficient to the adder 23-1.
The adder 23-1 adds the inverse orthogonal transformed signal obtained from the inverse orthogonal transformer 18-1 and the prediction signal obtained from the inter-frame/intra-frame predictor 22-1 to generate a locally decoded signal, and stores the generated signal in the memory 21-1.
The inter-frame/intra-frame predictor 22-1 may be provided with a switch (not illustrated) for switching between intra-frame prediction and inter-frame prediction.
[Intra-Frame Prediction]
The inter-frame/intra-frame predictor 22-1 predicts an original signal by extrapolation using decoded adjacent pixel information regarding the first component signal, generates a prediction signal, and sends the generated prediction signal to the subtracter 13-1 and the adder 23-1. Operations of other components can be considered to be the same as those of the case of the inter-frame prediction. For example, as an intra-frame prediction method, an encoding method for H.264 uses decoded adjacent pixel information as a reference signal and predicts an applicable component signal by extrapolation (e.g., see Patent Literature 1).